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Man seems unable to retain different information from different senses Of 
channels simultaneously; one channel gains full attention. However. :t ss hypotnesized 
that if the message elements arriving simultaneously from audio and visual channels 
are redundant, man will retain the information. An attempt was made to measure 
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redundancy in the audio and visual elements and in their relationship by showing a film 
clipping to groups who recorded their recall. Confidence can be placed in the two 



following hypothoses; the recall of the visual element in a message where audio and 
visual elements are presented simultaneously is positively related to the visual 
redundancy: the recall of both audio and visual elements which are presented 
simultaneously in a message is positively related lo the redundancy in the audio-visual 
relationships. The other four hypotheses, also dealing with the relationship of 
audio-visual element recall to redundancy, favored the hypotheses; but the results 
were not conclusive. These findings imply that audiences lose much television and film 
information. (MM) 
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I. SUMMARY 



The presumed communicative efficacy of simultaneous 
words and pictures does not always hold in practice. 
Travers tentatively suggests that 'for almost all instruc- 
tional circumstances and purposes, single channel communi- 
cation is most appropriate. Others have discovered 
evidence that an interference may develop between picture 
and word negating the desired response to the message. 
There is also the suggestion, by Broadbent, of a natural 
limitation on the cognitive capacity to receive informa- 
tion from two sensory channels simultaneously. 



Broadbent 
the mass 

ism so that mtormat 
orderly fashion to the 



screening 

organ- 



Liva 



proposes an internal system for 

input which arrives at the periphery of ^ the 
at in tormatiun is transmitted selectively a* 

central nervous system. The in- 
terior" " filtering" system accounts for the reduction of 
the mass input. The selected information then travels 
through the cognitive utilization channel successively; 
the information from only one channel at a time. However, 
if the utilization channel is clear at any time, other 
information may be drawn from a short term storage system. 
This is how information coming simultaneously from 
sensory channels may enter the utilization system. 



two 



It is hypothesized that, if message elements arriving 
simultaneously from audio and visual channels must enter 
the cognitive utilization system successively, the more 
the message can be "compressed," the greater the likeli- 
hood that the entire message will be processed. Sequential 
redundancy, as described by Shannon and Weaver, and syn 
chronous redundancy, as described by Osgood and Sebeok, 
would represent an index of "compressibility." 



An attempt was made to measure redundancy in the audio 
elements of a motion picture film clip, in the visual ele- 
ments and in the relationship between the two. Each of 
these measures was based on a separate group of subjects. 

A fourth group saw and heard the entire message. Immedi- 
ate recall of both audio and visual elements was recorded. 
The audio element was defined as a simple sentence; the 
visual element as a single camera "shot." Audio and visual 
elements were precisely opposed to form audio-visual units. 
Recall responses by element and by unit were correlate 
with the redundancy indices for the unit. 
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The following specific relationships were hypothesized. 

1) The recall of the audio element in a message where 
audio and visual elements are presented simultaneously is 
positively related to audio redundancy. 

9 ) The recall of the visual element in a message where 
audio and visual elements are presented simultaneously is 
positively related to visual redundancy. 

3) The recall of both audio and visual elements whieh 
are presented simultaneously in a message is positive y 
related to audio redundancy. 

4) The recall of both audio and visual elements which 
are presented simultaneously in a message is posi ive y 
related to visual redundancy. 

5) The recall of both audio and visual elements which 
are presented simultaneously in a message is positive y 
related to redundancy in the audio-visual relationship. 

6) The recall of both audio and visual elements which 
are presented simultaneously in a message is ^est pre ic e 
by a multiple regression equation based on audio redun y, 
visual redundancy and redundancy in the audio-visual rela 

tionship. 

All results were in the hypothesized direction. While 
significance levels were not predetermined, for the r 
tionships hypothesized in two and five above, p < .uo. 
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II . THE PROBLEM AND RELATED RESEARCH 



Introduction 



Electronic media such as television and sound rru^^ 
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picture films have the potential for transmitting complex 
audio and visual message elements simultaneously through 
multiple sensory channels. The audiovisualist, the ele 
vision journalist, advertising art director, film maker, 
and others concerned with pictorial presentation in one 
form or another, are prone to believe that this word 
picture complex communicates more effectively than words 
alone. But the presumed communicative efficacy of words 
and pictures does not always hold in practice. In some 
cases, under some conditions, a single channel is more 
effective. On the basis of a comprehensive survey of the 
literature and initial exploratory studies of multiple 
channel communication, Travers tentatively suggests that 
for almost all instructional circumstances and purposes, 
sinqle channel communication is most appropriate. koOJ 
Others have discovered evidence that an interference may 
develop between picture and word negating the desired re 
sponse to the message. There is also the suggestion of a 
natural limitation on the cognitive capacity to receive in 
formation from two sensory channels simultaneously. 

Seldom in audio-visual research, however, are the ele- 
ments of audio-visual communication isolated to determine 
their relationship to the general effect. It is the pur 
pose of this study to examine the major components the 
audio and the visual elements— and the synchronous relation- 
ship between the components. More specif ical y, o Y 

a rharacteristic (redundancy) which may be determined for 
each of the two components and for the synchronous relation- 
ship between the components, and relate this c arac en 
to one kind of effect (recall) . 



The Problem 



There is a long tradition of intuitive, empirical and 
theoretical support for the superiority of word-picture 
communication. Some of this support will be presented here, 
briefly, as well as a more lengthy treatment of the excep- 
tions. The latter are less commonly discussed. 



in Volume I. Number 1 of the psychological Review 

(1894), Munsterberg concludes a series of ^t^two senses 
savinq* "A series of presentations offered to 
at th! same time is much more easily reproduced than if 
given only to sight or only to hearing. (23:37) 

Communication practitioners have also «®]j a sJ£ed ^ he 
power of the word-picture combination. In 1922. Walt 
Lippmann wrote: " - - . it is probably true tnat • • • 

idea is lucid for practical decision until i- has visual 
or tactile value." (17:162) Wilson Hicks says: Pictures 

and words together perform a more effective function than 
either can per form alone." (1:122) An audio-visual text- 
book suggests that "... words with pictures are usually 
better than either words or pictures alone. (34.75) 



audio-visual literature support 
first is the cue summation theory 
stimulus cues in each channel are 
It is stated somewhat more 
"When the cues elicit the same 
or different responses in proper 
summate to yield increased effec- 



Two theories in the 
these conclusions. The 
which suggests that the 
additive in their effect 
formally by Miller et_ a_l 
responses simultaneously 
succession, they should 
tiveness." (20:78) 

A second theoretical position attributes a reinforce- 
ment effect to related stimuli presented simultaneously in 
twS channels. Cropper illustrates with this example: 

The demonstration of a solid object expanding as it 
is being heated can serve to cue or prompt the verbal, 
response, "expands." A second use of the demonstra 

tion might involve asking the student ^ f ” ul d occur 
heat is actually applied, to predict what would occur 

if heat were applied to the object. The v 1 ® 33 , ^ per 
ceived expansion following the application of heat 
would serve to provide reinforcement or confirm. 3 £ 
for the student. It would indicate to him that his 
response, "it will expand," «as correct. Its function 
here is similar to that provided by verbally tell g 
the student "it expands" or by verbally telling him 
that he is correct. (11:77) 

When the audio and visual channels are closely relate^ 
each oresents cues which elicit responses that are con 
firmed (reinforced) almost immediately in the other chan 
ne™ The learning theories of Hull, Skinner and others 
consider reinforcement essential to learning. ( 

Skinner! in particular, has demonstrated that -mediate 
confirmatory reinforcement enhances learning. (16) 
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Phrased in another way by May and Lumsdaine the 
theory states: " . . non-verbal stimuli ... when con— 

bined with the verbal explanations [form a] total pattern 
[which] has more power than words alor.e to evoke the de- 
sired responses. The non-verbal stimuli serve as rein- 
forcers in the Pavlovian sense of reinforcement." (18:166) 

Unfortunately, audio and visual cues do not always 
summate or reinforce each other to provide the desired 
effect in terms of the various learning criteria. Both 
the cue summation and reinforcement theories acknowledge 
that cues in each of the two channels may interfere with 
each other. 

Miller et_ ad . , note that contrary to a summation 
effect: "When the cues elicit incompatible responses, 

they should produce conflict and interference." (20:78) 
Gropper suggests that under certain conditions reinforce- 
ment is not likely: "When more than one response is under 

the control of a stimulus, the more likely is less effi- 
cient learning to occur as the result of response interfer- 
ence or competition." (11:80) 

Interference has been observed in some of the appli- 
cations of multiple channel communication in educational 
and informational media. Lumsdaine and Gladstone used 
two sound filmstrips to teach the old military phonetic 
alphabet. One presentation was "fancy" — "cartoon illus- 
trations accompanied by musical or other sound effects 
and/or by humorous comments by the narrator." This ver- 
sion might be considered to have a high level of informa- 
tion compared to the "simple" production which included 
only the printed words on a neutral background as the 
narrator pronounced them. Learning occurred as a result 
of both productions, but the "fancy" presentation was less 
effective. The attention gaining devices included in the 
"fancy" version did in fact draw attention, but this inter- 
fered with the efficiency of the learning. (18) 

In a film study, Neu developed four variations of a 
"clear, straight-forward" instructional film: 1) visual 

attention-getting devices, relevant, 2) visual devices, 
irrelevant, 3) auditory attention devices, relevant, and 
4) auditory devices, irrelevant. The relevancy of an 
attention-getting device in either channel was determined 
by its relationship to the intended message. Neu found 
some evidence that the basic film was most effective. 

For some of his populations the basic version, with no 
special devices to attract attention, was accompanied by 
the highest scores . The relevant and irrelevant attention 
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devices added to the information level of the message but 
not to the effectiveness of the film. (25) 

In another film study, Zuckerman experimented with 
levels of information in the sound track of a film find- 
ing that "... verbalization may be increased to a point 
where it interferes with, and actually reduces, learning." 
(36) 



Deutschmann, Barrow and McMillan measured the learning 
of both relevant and irrelevant information from film and 
"natural laboratory" presentations. They assumed that the 
more confined film presentation would be most efficient. 
"Channel efficiency" scores were determined for each sub- 
ject by deducting the percentage of correct irrelevant 
items from the percentage of correct relevant items. 

Channel efficiency scores were higher for the film pre- 
sentation, as predicted, suggesting to the authors that 
there were in fact interferences in the natural laboratory 
which could be screened out by the motion picture camera's 
frame. (9) Here again, is an indication that a limitation 
on the information level of the message enhances learning. 

In an attempt to estimate the relative influence of 
pictorial elements on interest and learning in television 
news, Hazard analyzed the three components of most news- 
casts: 1) man-on -camera, 2) still photographs, and 3) 

newsfilm. A measure of information gain established no 
significant difference between the three types of presen- 
tation, but the data disclose the highest gains in infor- 
mation for the man-on-camera unaided by still or motion 
pictures. (14) 

A series of three experiments was undertaken by Hart- 
man to assess the effect of multiple channel communication 
(audio, print, pictorial) . Over— all, the presentations 
including the pictorial channel were least effective. In 
discussion of the first experiment, which provided the 
best opportunity to check interference, Hartman states: 

"If the relation between the pictorial and verbal channels 
had been one of non-interference, the addition of the 
pictorial channels to the presentation would neither add 
not detract, and the results should be essentially similar 
to those obtained with the verbal presentations." (13:33) 
Instead, learning was at the lowest level when the picture 
channel was added. 

Why do we find evidence of interference? 
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Related Theory and Research 



A plausible explanation of interference, or the dif- 
ficulty sometimes experienced in attempting to simultane- 
ously use multiple channels for communication, is offered 
in an interesting historical account by Sumstine, writing 
in 1918: 

Several years ago, it was customary to have lecturers 
describe the motion pictures in the theaters; but at 
present they have discontinued. Several managers of 
motion picture theaters were asked the reasons for 
such discontinuance. Each one gave the same answer . 
The patrons objected to the combination, saying that 
they could not look and listen at the same time. 
(29:237) 

There is evidence in psychological research that sup- 
ports the finding of the movie theater owners. Audio- 
visual information is received in separate sensory chan- 
nels and must be processed by the organism. Whether two 
sensory channels, activated simultaneously, have access 
to the central cognitive system at the same time, is an 
essentially relevant question which is not usually asked. 
It is generally assumed that both audio and visual ele- 
ments of film or television messages are received and 
utilized. Recent studies, however, have cast some doubt 
on this assumption. 

Reviewing the Mowbray and Broadbent experiments to 
be discussed below, Berlyne notes one of the reasons for 
interference. He calls it occlusion: 

. . . an organism can do only a limited number of 

things at once, even of things no two of which are 
mutually exclusive. The capacity of the brain, 
therefore, must be setting limits to the efficiency 
with which information from two simultaneously active 
sources is utilized. (2:13) 

A tradition of research has focused on the possibil- 
ity of the divu.-'on, or splitting, of attention between 
simultaneous perceptual tasks. Division of attention 
might more appropriately be called division of attention 
in cognition , to indicate that attention is measured by 
some cognitive act related to the exposure to particular 
stimuli. The general conclusion is that the division of 
attention seems to be nearly impossible. (35) Where ex- 
perimental subjects appear able to divide attention, it is 
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assumed that an essentially unitary attention has been 
alternated between the two perceptual tasks. 

In this tradition, Mowbray's research is most recent 
and most relevant to the simultaneous audio-visual pre- 
sentation of information. By presenting different prose 
passages simultaneously in auditory and visual channels, 
Mowbray found that responses to subsequent questions on 
the passages did not rise above a chance level for one o 
the channels, except for the condition of easy passages in 
both channels. He concludes that: 

... as alternation of attention becomes increasing- 
ly more difficult, the possibility of performing 
adequately both of two simultaneous perceptual tasks 
becomes more remote. This evidence tends to suppor 
the thesis that attention is unitary, at least with 
respect to complex, cognitive acts. . . . Iclsofii 

Because of the timing of the visual presentation, sub 
jects reported being aware of attending to the audio pes 
saqe after assimilating the exposed visual passage. 
Apparently there was enough time to attempt to shift atten 
tion prior to exposure of the next visual element. An im- 
portant point is that the subjects attempted to perceive 
the material successively rather than simultaneously. 



Another experiment by Mowbray further reduced the 
possibility of alternation of attention. (22) This time 
subjects were presented two different pairs of map coor 
dinates simultaneously, one pair in the auditory mode and 
one pair in the visual mode. They were to locate details 
on maps using the pairs of coordinates. The simultane 
ously presented pairs of different coordinates were ran- 
domly imbedded within irrelevant words which were the same 
in both channels. All audio elements were paced to match 
the visual elements. Half of the subjects were instructed 
that each set of relevant coordinates would be presented 
simultaneously with another set of relevant coordinates. 
The other subjects were "uninstructed." Subjects were 
encouraged to guess at the locations if they had any rea- 
sonable basis. The analysis indicated that in a few c 
both sets of coordinates were used correctly. These, how- 
ever, were below the theoretical frequencies expected by 
chance using the most conservative estimate of chance 
levels. The "instructed" group made no more successful 
completions of both simultaneously presented locations ^ 
than the "uninstructed" group. Among the "uninstructed, 
no subject was able to tell the experimenter the special 
simultaneous condition of the relevant items. Almost all 
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denied being aware of it, and many were vehement in their 
disbelief. Again, Mowbray concludes that simultaneous 
multiple-channel perception is not possible: All o 

results from this experiment suggest that complex percp- 
tions involving language cannot be affected by different 
sensory modes at the same time." (22:92) 

Two demonstrations of this result also appear in 
pilot studies by the present author. In the first study, 
each of a series of four television news stories on film 
were projected for subjects. Immediately afterviewing, 
subjects were asked to write down as much of the visual 
and audio information as they could recall. y 

these responses, the stories were broken down into the 
simultaneously opposed audio and visual elemen 
statements of fact in the sound track and scenes ‘nSaae 
visual portion. For all four stories, when the percentage 
of subjects recalling an audio element, was high the P er 
centage recalling its opposing visual element was very low. 
When subjects recalled visual elements, audio recall was 
low. There seemed to be an alternation of attention be- 
tween the audio and visual elements. This held f ° r . x £ t’ 
vidual subjects as well as the group. (See Appendix A.) 

The same experiment was attempted with a short fii™ 
clip adapted from an instructional film produced by W 1 
Disney. (30) Again, there was a marked a J- te ^ n ^ lon ,g" 
recall between opposing audio and visual elements. (See 
Appendix B.) Recall of one was high at the expense of 
the other, in most cases, suggesting Mowbray s conclusion 
that complex perceptions cannot be affected by different 
sensory modes simultaneously utilized. 

These findings indicate that, at least for complex 

materials, attention seems to be unitary • Br ° ad ^ 8 " ( 
however, has demonstrated the human capacity to utilize 
two sets of information delivered simultaneously to two 
sensory channels. At the same time he agrees that atten- 
tion (in cognition) is unitary. 

Presenting two different sets of digits each 
simultaneously, Broadbent found subjects able to repe 
all of the digits. One ear would receive digits such a 
4-2-°; and the other 8-6-3. The "natural response" was to 
report 4-2-7, 3-6-3 (or 8-6-3, 4-2-7). Subjects never re- 
ported the simultaneous pairs in temporal sequence as they 
££e transmitted, i.e.. 8-4, 6-2, 3-7. In fact, when sub- 
jects were told to respond in this latter order, they had 
great difficulty, some finding it impossible. 
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To explain these findings Broadbent proposes that 
some mechanism must operate to receive items simultaneous- 
ly, with still another mechanism operating which will only 
handle items successively. The former he calls the S sys- 
tem and the latter, the P system. The S system receives 
all of the sensory input to the organism, but the ultimate 
perceptual processing system (P) , which is unitary, admits 
information only from a single channel at a time. Other 
information may be held in storage in the S system, how- 
ever, for brief periods. The dual system accounts for the 
ability of subjects to respond to simultaneous inputs of 
different information and also the successive manner of 
that response. (7) 

The duration of the storage period was tested in ano- 
ther experiment. (4) In this instance, the P system was 
kept occupied for various periods of time after the arrival 
of information in another channel. A stream of six digits 
would be presented to one ear. Two different digits would 
be presented simultaneously to the other ear with the first 
two of the six. For another set of six digits, the second 
ear would receive different digits simultaneously with the 
third and fourth. In the final condition, the two simul- 
taneous digits would be paired with the last two digits in 
the stream of six. Subjects were told to repeat first the 
six, then the two. This could be done quite well for the 
last condition, but much less well for the conditions 
where the two were presented earlier. Additionally, the 
latter of the two digits was more likely to be recalled 
than the first — it had been in storage for a shorter time. 
Broadbent concludes that " . . .if the two digits were in 

the S system for more than a second or so before their ad- 
mission to the P system they were less efficiently recalled. 
(5:227) 

To be sure that the capacity for storage was not pe- 
culiar to the auditory channels, Broadbent designed another 
experiment using the audio and visual channels. (6) This, 
of course, is of particular significance in the present 

context . 

In one condition, two different digits were presented 
simultaneously, one to the right ear and the other to the 
eyes. After the third such pair of digits, the subject 
was to write them down in any order. In another condition, 
subjects received three digits successively in the right 
ear as before, but the visual digits were exposed all at 
once, appearing as soon as the first aural digit and dis- 
appearing after the third. 
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Subjects were able to correctly recall the digits 
from both channels fairly consistently; as well or better 
than control situations where the digits were presented 
binaurally , successively in the visual channel ana succes- 
sively in the auditory channel. This establishes that the 
capacity for reception of simultaneous information is not 
specific to the binaural condition. And, it was aga n 
found to hold that all information from one 
produced before the other. This was the case in 
correct responses. Again we have the suggestion by Broad 
bent that "adequate response to simultaneous stimuli w n 
achieved by delaying all the information on one channel. 
(6-147) It is interesting to note that while subject 
showed some tendency to consistently report from one sen 
sory channel first, they divided equally in this prefe 
with half reporting visual information fir , 
half favoring auditory information. 

To review, Broadbent proposes an internal system for 
screening the mass input which arrives at the periphery of 
the orqanism so that information is transmitted selective 
S and 9 in orderly fashion to the central nervous systn. 

The interior "filtering" system, as Broadbent descri 
the P and S systems working tohether, accounts for 
duction of the mass input. The selected information travels 
through the final system single file, one might say, or 
successively; the information from only one cha "" e ^ 
time. However, if the single charnel is dear at any tim , 
information may be drawn from the short term storage sy 
tem This is how information coming simultaneously from 
the 'two sensory channels appears to enter the P system 
According to Broadbent, the storage system is only °H eis 
tive when there is an overload of information. Storage is 
not necessary when information arrives successively. On 
the other hand, if the main channel is occupied with the 
input of one sensory channel beyond the maximum storage 
time of two or three seconds, the information in storage 
from another channel is lost. 

Travers has used the Broadbent constructs to schema- 
ticize the audio-visual information transmission process 
and to more explicitly indicate what he calls information 
"compression." (See Figure 1.) 

"Compression" refers to any process within the nerv- 
ous system which reduces redundancy. (33: 5. ^3} At tn „ 

outset naturally occurring events and some precompr 
or*" precoded, stimuli are received at the receptor level 
whereupon it is compressed by the sense organs ' ^ ± x 

peripheral nervous system. For instance, visual material 
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is compressed to the determining boundaries which suffi 
ciently carry the message. At another level, compression 
takes place through categorization. This process is de- 
fined as the "rendering of discriminably different stimuli 
as equivalent." (33:4.08) One would assume that the sys- 
tem generalizes stimuli where certain discriminations do 
not seem to be important or necessary. Travers also uses 
the term "categorizing" interchangeably with "coding." 

The latter is considered in information theory terms as a 
reduction in the number of signals needed to relay a given 
amount of information." (33:4.09) 

In other terms, the process might be called "recoding" 
or "chunking." Miller defines the process as organizing 
or grouping input into familiar units or chunks . 

The input is given in a code that contains many chunks 
with few bits per chunk. The operator recodes the in- 
put into another code that contains fewer chunks with 
more bits per chunk. There are many ways to do this 
recoding, but probably the simplest is to group the 
input events, apply a new name to the group, and then 
remember the new name rather than the original input 
events. (33:109) 

Returning to the Travers schematic, the compressed 
"stimulus traces," or chunks, then enter a monitoring or 
selecting mechanism in the schematic. This mechanism pre- 
vents the utilization channel from becoming overloaded. 
Otherwise the utilization system would become jammed with 
information. The function is to limit the "input of infor- 
mation so that only manageable quantities have access to 
the higher centers." (33:9.08) This corresponds to Broad- 
bent's filtering concept. Information may be stored at 
this point for later selection, providing the stimulus 
"trace" is selected within two or three seconds. 

The schematic suggests an answer to one of the ques- 
tions we have raised — why does interference occur? In the 
process, we also answer another question which is raised 
by the answer to the first — why does interference not 
always occur? In other words, we may infer, from the 
schematic, conditions under which audio-visual communica- 
tion may sometimes be cognitively received and other times 
not received. 



III. THE HYPOTHESES 



The model of the single channel perceptual system 
seems to suggest that compression, or "redundancy," is a 
major factor in the capacity of the system to receive in- 
formation simultaneously from the audio and visual elements 
of a multiple channel message. Redundancy may be in the 
audio element, in the visual element and in the relation- 
ship between the audio and visual elements. In this sec- 
tion we wills 1) discuss the forms of redundancy in motion 
pictures and television, 2) relate redundancy to the 
Broadbent-Travers concept of the single channel perceptual 
system, and 3) put forth a set of hypotheses based on the 
apparent function of redundancy in the perceptual system. 



Redundancy 



In their paper on synchronic psycholinguistics, Osgood 
and Sebeok discuss redundancy within audio and visual chan- 
nels, and between the channels. (27) While their terms 
relate specifically to inter-personal communication, they 
seem appropriate, with some adaptation, to other forms of 
communication. Osgood and .lebeok identify bands over 
which messages travel synchronously; the vocal-auditory 
band, the visual-gestural band and the manipulational- 
situational band. In audio-visual communication, certain 
elements of the interpersonal vocal-auditory band may be 
missing because of the technical limits of the recording 
and reproducing equipment. Distortion, or missing fre- 
quencies, however, if detectable, are not often critical. 

The vocal-auditory band, then, is comparable to the audio 
element of the audio-visual message. 

The visual-gestural band is sometimes present in audio- 
visual communication. As in man-on-camera presentations. 
Again, the fidelity may suffer somewhat in the reproduction 
processes, but the result still has most of the character- 
istics of the interpersonal form. 

The manipulational-situational band is construed by 
Osgood and Sebeok to include graphic communication, which, 
one could assume, would include photographic or electronic 
representation. This band is comparable to the most common 
type of visual element in audio-visual communication. 
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Osgood and Sebeok refer to the relationship between 
the vocal-auditory, visual-gestural and manipulational- 
situational bands as between band organization. 

The authors acknowledge the evidence for sequential 
redundancy for both audio and visual communication. This 
is related in the literature of information theory and 
kinesics. But they also propose a synchronic redundancy, 
within and between bands. 

The notion of sequential redundancy between parts of 
a message as serially unreeled is now a fairly common 
one, particularly as a result of the work of Shannon, 
Miller, and others. The notion that there can also 
be synchronic redundancy among simultaneous events 
within the same band or between bands is less familiar 
but equally reasonable. Both linguists and informa- 
tion theorists have taken cognizance of redundancies 
within the linguistic band per se, the former observ- 
ing that phonemes are for the most part overdetermined 
(in terms of clusters of correlated features) and the 
latter reporting that one can experimentally cut out 
50 percent or more of the total information in the 
auditory channel without seriously hampering intel- 
ligibility. There is also redundancy between dis- 
cretely and conti \uously coded signals in the vocal- 
auditory band — witness how stress is typically 
accompanied by lengthening of vowels, how stress and 
raised pitch tend to go together, and so forth. 

(27:88) 

These authors do not discuss redundancy in the mani- 
pulational-situational band, but Osgood, in a later article, 
(26:28) notes the identification by Harrison and MacLean 
of synchronic redundancy in graphic facial representation. 

The latter used cartoon faces with simple features as 
stimuli and asked subjects to relate descriptive adjectives 
to the faces. Such features as medially upturned brows 
and down-curved mouth, by themselves, tended to elicit the 
same responses. Combining the features does not bring 
significant new responses. "When combined, the elements 
tend to elicit the same responses again, only more strongly." 
(12:15) 

This is an example of synchronic redundancy within the 
manipulational-situational band. It deals with static 
visual content which is present in audio-visual communica- 
tion even if our unit of analysis is the "shot" in motion 
pictures or television. 
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Another form of redundancy in the static visual pat- 
tern is described by Garner: 

Redundancy means that fewer patterns exist than could 
exist, and when we say that a particular finite se- 
quence is redundant we are simply saying that this 
sequence is one of many which could occur, but that 
the number which could occur is far less than the 
total number of such sequences which could occur if 
there were no redundancy. Thus a unique pattern is 
judged to be redundant because it seems to be one of 
a smaller class of patterns than does the figure 
judged non -redundant , irregular, complex, etc. . . . 

What I am suggesting, in summary, is that a judgment 
of the redundancy of a unique figure (or judgment of 
other characteristics presumably related to redundancy) 
is not really a judgment of the redundancy of the 
figure itself (which is a meaningless judgment) but 
is, rather, a judgment of the redundancy of the 
particular subset of figures from which the unique 
figure is presumed to have come. (10:203-205) 

Garner uses the example of a vertical line which would 
be judged more redundant than a slanting line, because 
there are a smaller number of vertical lines possible than 
slanting lines. He would also consider the square more 
redundant than the trapezoid because there are more ways 
to construct a trapezoid than a square. 

Redundancy in this sense may be related to the in- 
finitely more complex static picture. If the picture 
represents a specific instance of a small subset of experi- 
ences, it is more redundant than the picture which repre- 
sents a more general set of visual experiences. This is 
another way of viewing its level of abstraction. 

In addition to synchronous redundancy in a static 
situation within a motion picture or television shot, we 
must also contend with sequential redundancy. Usually some 
action occurs. The probabilities of subsequent elements 
in an action, once it has begun, are contingent upon the 
character of the action. Once the bow has been drawn, the 
shot of the arrow would likely be redundant. 

In the foregoing, we have discussed sequential and 
synchronic redundancy in the audio and visual elements (or 
bands) of the audio-visual message, but not redundancy in 
the audio and visual relationship — the between band organi- 
zation. Osgood and Sebeok says: “Redundancy between bands, 

e.g., between vocal— auditory and gestural— visual bands, has 
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been for the most part neglected. ..." (27:88) They 
identify, however, from "informal observation," the extremes 
in redundancy between bands: 

(1) synchronic complementation , the usual situation 
in which gestural signals have the same significance 
as vocal signals and hence complement one another; 

(2) synchronic contrast , the more informational situ- 
ation in which gestural and vocal signals have dif- 
ferent (usually opposed) significance and hence change 
each other in some fashion. (27:88) 

The redundant case, synchronic complementation, is 
illustrated by the vocal "No" and the shaking of the head. 
Closer to our interest in audio-visual film or television 
communication is the example of synchronic redundancy be- 
tween the manipulational-situational and the vocal-auditory 
bands, e.g., disgramming or doodling on a pad to facilitate 
communication . 

The simplest case of non-redundant synchronic contrast 
mig^it be illustrated by the vocal "No" and the head nodding 
up and down. More commonly synchronic contrast is found in 
sarcasm, where the gestural-visual band denies the vocal- 
auditory band. Osgood and Sebeok suggest that at another 
level, synchronic contrast may be represented by an irrele- 
vant relationship between two bands. 

Birdwhistell calls this form of contrast "metaincon- 
gruence," which occurs "when an utterance has one contextual 
meaning while the accompanying body motion conveys a dif- 
ferent meaning (for example, the actors in television com- 
mercials, while verbally praising a product to the skies, 
can sometimes be seen rejecting it in their body motions) ." 
(3:15) 

While an irrelevant or incongruent relationship is not 
common and even difficult to achieve in the interpersonal 
communication situation, it is a very likely occurrence in 
audio-visual communication where the relationship is con- 
structed rather than spontaneous. 

In this section we have attempted to describe the 
various forms of redundancy in audio-visual communication, 
in particular, motion picture films and television — redun- 
dancy in the audio channel, in the visual channel and 
redundancy in the relationship between the two. The task 
remains to relate redundancy to the model of the single 
channel perception system. 
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Redundancy and the Single Channel 
Perception System 



As Travers uses the concept of compression in the 
model of the information transmission process, the more 
redundant the message the greater the compressability of 
the message elements. The more the message may be com- 
pressed, the more it is likely to stay within the capacity 
of a single channel utilization system. Compressed infor- 
mation from one channel will pass through the utilization 
channel more rapidly. Other information, simultaneously 
received from another channel which has been stored briefly, 
will then more likely gain access to the utilization channel 
before it has dissipated. It will be recalled that accord- 
ing to Broadbent's experimental findings, the storage life 
is very brief, a matter of one or two seconds. 

This application of the single channel model, with its 
storage system, was tested in the previously-mentioned 
pilot study by the present author using an adaptation of a 
short film clip from a Walt Disney educational production. 
(30) The visual elements in this analysis were scenes or 
"shots." The audio elements were opposing scatements in 
the sound track. One group of subjects was asked to judge 
each of the visual elements in terms of the subjective 
degree of certainty about the content of the elements . 

A five-point scale with the polar terms "certain" and 
"uncertain" was used for this purpose. The same scale was 
used with another group of subjects to determine the rela- 
tive "certainty" of each of the audio elements. On the 
same scale, a third group judged the relationship between 
the two channels. 

It was assumed that degree of perceived "certainty" 
would be a function of the degree of synchronic redundancy. 
The scale should reflect degrees on a continuum between 
certainty and entropy. 

The measures of redundancy were subsequently corre- 
lated with measures of immediate recall for both audio and 
visual elements taken on still another group. 

In terms of the Broadbent and Travers models of the 
perceptual system, it was predicted that in the case of 
high visual redundancy, the system would be more likely 
to accommodate the visual element, since the element could 
be compressed or coded. It would also be more likely for 
both simultaneous audio and visual elements to be recalled 
since the visual compression would allow more time for the 



momentarily stored audio information to be brought into 
the utilization channel (p) , if this were the order of 
entry. if the audio elements were to enter the utilization 
channel first, the compressed visual information would be 
more likely to fit into any remaining time. 

The same rationale could be presented for the predic- 
tion that high audio redundancy would be related to high 
recall and to the recall of both elements in a simultane- 
ously presented audio-visual unit. 

Finally, it was predicted that the cross— channel 
relationship would be positively related to the recall of 
both simultaneously presented elements. If the element in 
storage interacts with the element which has already passed 
into the utilization channel, then it is more likely to, 
itself, enter the channel. in other words, the stored ele- 
ment may become redundant or more certain by its relation- 
ship to the other element which has already entered the 
cognitive system. 

An analysis of the data from this pilot study indi- 
cates the strongest relationships are with visual redundancy 
and recall. Visual redundancy seems to be related positive- 
ly to the recall of both elements and to a lesser degree to 
the recall of visual elements. There is no clear relation- 
ship between audio redundancy and recall. One reason for 
the failure to obtain the predicted result here lies in 
what was apparently the incapacity of the instrument for 
discrimination between audio elements. There was little 
variance among elements. 

Cross-channel relationships seem to be slightly related 
to the recall of both channels as predicted. The relation- 
ship might have been stronger if the film had been presented 
to the subjects rather than a verbal description of the 
visual elements. (See Appendix C.) 

There are three difficulties in interpreting the data 
above. Two have already been mentioned. First, either 
the elements of the audio channel did not vary enough in 
redundancy, or the instrument used to measure redundancy 
was not sensitive enough. Secondly, redundancy in the re- 
lationship between channels was not measured by using the 
or igi na l materials, but rather a printed verbal description 
of the visual channel and a printed audio script . A better 
estimate of the relationship would have been obtained by 

using the same sensory channels as the original presenta- 
tion . 
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Finally , it should he noted that there was some oppor- 
tunity for alternation of attention between channels. For 
all visual elements, the opposing audio element was slightly 
shorter, by a second or two. 

Nevertheless, the empirical data from the pilot study 
lend some support to the contention that redundancy in the 
visual channel and redundancy in the relationship between 
channels is related to recall. And, our theory would pre- 
dict the same effect for redundancy in the audio channel. 



Hypotheses 



This would lead us to hypothesize the following rela- 
tionships: 

^ The r ecall of the audio element in a message where 
audio and vis ual elements are presented simultaneously is 
positively related to audio redundancy . 

2) The recall of the visual element in a message where 
audio and vis ual elements are presented simultaneously is 
positivel y related to visual redundancy . 

These two hypotheses are derived from the theoretic 
model. If audio redundancy is high, the recall of the 
audio element should also be high, since the redundant ele- 
ment is more easily compressed. The same would hold for 
visual redundancy with visual recall. There is also a kind 
of repetition inherent in redundancy which should have the 
effect of cue summation. In some studies, with unfamiliar 
forms and words, recall appears to be a function of fre- 
quency or exposure (8:252). 

3) The re call of both audio and visual elements which 
a.re presente d simultaneously in a message is positively 
related to audio redundancy . 

4) The recall of both audio and visual elements which 
are presente d simultaneously in a message is positively 
related to visual redundancy . 

According to the Broadbent theory as interpreted here, 
compression potential in one channel should increase the 
likelihood of both elements in an audio-visual unit being 
recalled. The element may pass through the utilization 
channel more rapidly, thus permitting the second element to 
be called up from storage before it has dissipated. Or, if 
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the element is selected for storage, it is, in its com- 
pressed form, more likely to fit into any remaining time 
after :he opposite element has passed the utilization 
channe L . 

In this case the effect of repetition in one element 
does not explain recall of both elements. 

5) The recall of both audio and visual elements which 
are presented simultaneously in a message is positively 
related to redundancy in the audio-visual relationship. 



If the elements of a simultaneously presented audio- 
visual unit are related, or redundant, the element in 
storage may become compressed after the element which has 
passed into cognition since it is repetitious and only its 
unique characteristics need enter the utilization channel. 
This latter process would be shorter in duration, with 
greater opportunity for any non -redundant information in 
the element to enter the utilization channel prior to dis- 
sipation or the next input. 

6! The recall of both audio and visual elements which 
are presented simultaneously in a message is best predicted 
by a multiple regression equation based on audio redundancy , 
visual redundancy and redundancy in the audio-visual rela - 
tionship . 

It is assumed that audio redundancy, visual redundancy, 
and audio-visual redundancy are independent and therefore 
not correlated by unit. Together, the three should each 
contribute independently to the prediction of the recall 
of both elements in a given unit. 

The terms of these hypotheses will be defined opera- 
tionally in the section that follows. 
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IV. METHODOLOGY 



To provide evidence for the validity of the hypothe- 
sized relationships, it is first necessary to identify 
meaningful audio and visual elements. These must be pre- 
cisely opposed temporally to form the audio-visual units. 
Recall of the elements of the message must be recorded. 
Indices of visual redundancy, audio redundancy c*.id redun- 
dancy in the audio-visual relationship for each of the 
elements and units must be obtained. These indices may 
then be correlated with recall of the elements to indicate 
the degree of relationship. 



Materials and Subjects 



An existent film was used as the basis for the audio- 
visual message. It was the same film clip as used in pilot 
studies mentioned in Sections II and III. The clip was taken 
from a Walt Disney color production, Switzerland , designed 
for social studies units at the upper elementary and secon- 
dary levels. The particular segment used described the 
cheesemaking procedures of a village cheese maker in the 
Swiss mountains. 

Some rewriting and editing was necessary to make each 
of the audio and visual elements match precisely. The 
final production was 40 seconds long with 10 scenes and 10 
sentences, each about four seconds in length. (See verbal 
description of scenes and the script, Appendix D.) 

Each visual element was a single scene in the film. 

More technically, as defined by the University Film Pro- 
ducers Association, the visual element would be considered 
a "shot" — the piece of film resulting from a single run of 
the camera. (24:27) The shots from the original film 
were edited precisely to the desired length by frame counts. 
Some shots in the original sequence were not used. This 
did not disrupt the continuity, however. There were no 
titles or closing credits. The experimental content was 
preceded and followed by opaque leader. 

The audio elements were simple sentences, some with 
modifying phrases. The tape was narration-only with no 
background music or natural sound. The new sound track 
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was recorded on magnetic tape in the Department of Tele- 
vision and Radio studios at Michigan State University. 
Technically, it would meet high standards. The narrator 
is the instructor of television and radio announcing at 
Michigan State who has had a number of years professional 
experience as an announcer. 

To project the film with sound in synchronization, 
the tape was cued to its starting point on a recorder. 

The proper volume was preset. The recorder was started at 
the moment the picture appeared on the screen. In all 
administrations, the synchronization was perfect. 

Since the film was designed for upper elementary and 
secondary levels, subjects were selected from within this 
range. They were high school students, mostly juniors, 
attending workshops in communication arts and in music at 
Michigan State University in the summer of 1966. 



Recall 



Seventy-four subjects were exposed to the film in syn- 
chronization with its narration. They were in separate 
groups of 30, 12 and 32. Subjects were told, as were all 
others participating in this project, that they were taking 
part in an ongoing research project of the College of 
Communication Arts, Michigan State University. All materials 
were so labeled. No other instructions were given at this 
time except to note that the film was only 40 seconds long 
and therefore it was necessary to pay close attention from 
the start. This was done in an attempt to have subjects 
immediately alert. All three groups viewed the film under 
approximately the same conditions, although at different 
times over a two-week period. The rooms were the same size 
and darkened to about the same light level. With the pro- 
jector and recorder located in the same place in each room, 
the image size and sound level were nearly identical in all 
rooms. There were no apparent distractions or disturbances 
during any showing, although in one room the screen had a 
small blemish in the lower right hand corner which appeared 
as a black mark within the picture. 

Immediately after the film, subjects were given a 
booklet which included the instructions and three pages 
for the required response. (See Appendix E.) The instruc- 
tions were read aloud as subjects read silently. They were 
asked to write as much as they could recall from the narra- 
tion and from the visual portion. The response pages were 



divided down the middle and lined. The space at the left 
was labeled for the visual, the right hand side was labeled 
for the narration. Subjects were told to write items down 
as they came to mind without regard to order or to the rela- 
tionship between the visual and narration columns. Five 
subjects did not respond in the manner requested. The forms 
for these subjects were voided, leaving a total of 69. 

The responses to each audio and visual element for 
each subject were coded by the author in a binary system; 
1-recall, 0-no recall. (See recall score sheet, Appendix F.) 
If the "essence" of an audio or visual element was mentioned 
it was scored as recalled. The mention had to be specific 
enough to be assigned to a particular element. Occasionally, 
a subject would be confused as to whether an item originated 
in the visual or the narration. In this case the item was 
scored for its proper channel. 

In the great majority of cases, it was not difficult to 
code the recall responses. The responses could be easily 
identified with their appropriate elements. In some in- 
stances, however, it was necessary to make a judgment as to 
whether enough information was included in a response to 
warrant coding it as recall. All responses were coded again 
by the author a week after the first coding to provide an 
indication of the reliability of the coding over time. 

There were 105 disagreements between the first coding and 
the second out of a possible 1380 (92.39 percent agreement). 
Each of the points of disagreement was then reconciled. 

A substantial proportion of these were errors in coding which 
simply rec[uired correction on the final code sheet. Others 
were the type where a somewhat arbitrary judgment was made. 
These were resolved after careful study of the response in 
question. No index of inter judge reliability was obtained. 
While this would have been desirable, the labor involved 
for additional coders would have exceeded the limits of 
what one might ask. There was, however, no opportunity for 
intentional or unintentional bias in favor of the hypothe- 
ses since all recall coding was done prior to the coding of 
redundancy indices. 

It was assumed that there would be a single predominant 
pattern of response — that subjects would tend to uniformly 
recall certain audio and visual elements and not others. 

This assumption is indeed critical. If subjects responded 
randomly in entirely individual patterns, there would be no 
point in the analysis. A factor analysis of persons was 
therefore performed to determine a response pattern. If a 
single or dominant pattern emerged, those subjects with the 
highest loadings on that factor would be the best indicators 
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of a general effect (of the message on recall) . The devi- 
ants from the pattern, if there were not too many, could be 
eliminated as "bad" items would in an item analysis. 
Subjects might deviate from a general pattern for a number 
of reasons such as a peculiar attraction to a particular 
element, momentary distractions, low level of attention, 
etc . 

The following scores were obtained from these data for 
all 69 subjects and also for the group with the highest 
factor loading on the dominant response pattern: 

a. the total number of subjects for each of the 10 
audio-visual units recalling both audio and visual 
elements — RECALL BOTH (audio=l, visual=l) . 

b. the total number of subjects for each of the 10 
audio-visual units recalling the audio element — 
AUDIO RECALL (audio=l, visual=l or 0) . 

c. the total number of subjects for each of the 10 
audio-visual units recalling the visual element — 
VISUAL RECALL (visual=l, audio=l or 0) . 



Audio Redundancy 



A device similar to Taylor's cloze procedure was used 
to measure sequential redundancy in the audio elements. 

The procedure involves systematic or random deletion of 
words in printed passages. Blank lines replace the deleted 
words. Subjects are asked to guess the missing words. If 
a passage is redundant, the context may provide cues to the 
missing words. The more redundant a passage, the higher 
the probability of correct responses. (32) 

Since our purpose was to determine the relative redun- 
dancy of audio elements — sentences — the usual cloze pro- 
cedure could not be employed. Instead of a fixed number of 
random deletions or a fixed ratio of systematic deletions 
such as every fifth word, different subjects responded to 
each word of the sentence, except the first. 

Another change in the usual form of cloze procedure 
was made to better accommodate the measure to audio pre- 
sentation. Only the words prior to the missing word were 
supplied to the subject. In cloze procedure, words that 
come after the blank also provide contextual cues . This 
seemed more appropriate for self-paced reading material 
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where the reader may scan the whole sentence and return to 
ponder the blank. The listener is not able to make as good 
use of after-the-blan>. contextual cues. Since the listener 
is forced to attend to an ongoing, externally-paced message, 
he is not as likely to be able to relate the after-the- 
blank context to the missing word. The present procedure 
of supplying only the words before the blank would also 
seem to better fit the concept of sequential probability or 
predictability which would be most crucial in a single chan- 
nel utilization system. 

With this procedure nc measure was possible for the 
first word of each sentence. At least the first word was 
necessary to provide a context. The longest sentence of 
the 10 had 13 words which meant 12 deletions. It was arbi- 
trarily decided to obtain at least three responses for each 
word. This required a total of 36 subjects. Each subject 
responded to one deletion in each of the 10 sentences. 

After all the required deletions had been made for a single 
sentence, they were randomly shuffled. All of the sentences 
were shuffled so that each subject received deletions which 
were randomly located within the sentences. The word 
deleted in the first sentence for a given subject might be 
the seventh; the deletion in the second sentence for the 
same subject might be the third, etc. 

The deleted sentences appeared in the same order as 
in the film script, each printed on a separate page. In 
oral instructions, subjects were told that they were to 
quess at the next word in the portion of a sentence on each 
page. After a few seconds to make the guess, the complete 
sentence was read by the administrator. This was done so 
that subjects could take advantage of any contextual cues 
from previous sentences which might contribute to the re- 
dundancy of subsequent material. (See instructions and 
sample instrument, Appendix G.) 

The instrument! were administered to two groups, one 
of 24 students and the other of 12, at different times in 
the sane week. "Correct" responses were the precise word, 
or a derivative thereof, used in the film script. Because 
some words in the shorter sentences had more than three 
deletions, a fraction was computed for each word based on 
the number of correct responses in relation to the number 
of possible correct responses. The fractions were added 
over all the deleted words in a sentence and divided by 
the number of words in the sentence less one. The result- 
ant figure was used as an index of redundancy for that 
sentence . 
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Visual Redundancy 



Rating scales similar to those described for "visual 
certainty in Section II were used to measure visual redun- 
dancy. it was established in the pilot study that subjects 
could discriminate between film scenes on the basis of a 
five-point scale with poles labeled "certain" and "uncertain." 

Subjects were instructed to rate a scene on the basis 

of the degree of certainty in their comprehension of the 
scene: 



CERTAIN : : : : UNCERTAIN . 



The first point was described as "certain, quite clear"; 
the second as fairly certain, fairly clear" ; the middle 
point as "not sure"; the fourth as "fairly uncertain"; and 
the fifth as "completely uncertain, unclear, confusing." 
(See Appendix H.) 

Each scene was projected on the screen followed by 
opaque leader . They were shown in the original order for 
the same reason that the audio elements were kept in the 
proper order , so that prior elements could contribute to 
the redundancy measures as the film progressed as would be 
the case in a natural situation. The projector was stopped 
for 10 seconds after each scene. Subjects were to rate 
the scene during this period. Thirty— three subjects parti- 
cipated. The scales were coded 4, 3, 2, 1, 0 from left to 
^■^■ 9 ht, from certain to uncertain. The total score over 
all subjects for each scene served as the index to its 
visual redundancy — the higher the score the higher the 
redundancy. 

As discussed in Section III the "certainty" scale is 
taken to represent a range between relative certainty and 
relative entropy . If one might think of the process of 
viewing a picture in two steps, the first would be an 
orientation response where the picture is scanned for 
familiar cues or points of reference. if there are enough 
of these, we might expect a second step, an identifying or 
recognition response to fairly high certainty. if the cues 
or referents are insufficient, unfamiliar or unclear, we 
*^9ht expect a longer orientation period and less certainty 
in the identification or recognition response. 



From the initial moment of the orienting response, to 
the time the scene content is satisfactorily identified, 
uncertainty or entropy which originates in the orienting 
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response is being reduced . If the elements or actions in 
a scene are not successfully identified, the entropy origi- 
nating in the orientation response continues, or may even 
increase. The latter would be the case if incompatible or 
incongruent cues are discovered. 

For our purposes, it was assumed that if enough non- 
conflicting visual cues were presented in a shot to provide 
some certainty in the identifying response, subjects would 
be able to indicate this on a scale representing certainty 
of comprehension. Response to fewer, inadequate, or con- 
flicting visual cues would register on the negative side 
of the same scale. Both sequential and synchronic redun- 
dancy would be reflected in the response. 



Audio-Visual Redundancy 



Redundancy in the audio-visual relationship for each 
audio-visual unit was determined by asking subjects to 
match the elements. The scenes were randomly ordered and 
separated by opaque leader. A different random order was 
established for the audio elements. These were printed on 
a single sheet of paper. Eacn sentence was preceded by a 
blank line. So that subjects would be thoroughly familiar 
with the sent nces, they were read orally twice by the 
administrator while the subjects read along silently. 

After a scene was projected it was numbered orally. 

The number was to be placed on the blank next to the sen- 
tence which most closely corresponded to the scene. Sub- 
jects were permitted to use a sentence more than once. 

They were told to guess if there were no obvious choice. 
Thirty subjects participated. 

A slight revision was made for a second administration. 
The instructions were changed to permit subjects to use 
more than one sentence for a scene if others also were appro- 
priate. This was to avoid the possibility of some sentences 
being excluded because others fit about as well. The in- 
struction read: "... write the [scene] number opposite 

the sentence that most closely corresponds with that scene 
and any other sentence that might also fit the scene. You 
therefore might have a particular scene number written in 
front of more than one sentence." Nineteen subjects 
participated in this administration. In both versions, sub- 
jects had 40 seconds to make their choices for each scene. 

At the end of this period they were warned to finish and 
look up at the screen for the next scene. (See Appendix I.) 
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The correct matches, according to the original script, 
were totaled over all subjects for each audio-visual unit. 
The higher the score for a unit, the greater the redundancy 
in the relationship between the audio and visual elements. 
If subjects were able to see something in one element that 
corresponded to another, and these were the elements which 
were actually matched in the film, then there is some 
redundancy in the relationship. It was assumed that the 
less agreement (the fewer correct matches) among subjects, 
the less obvious the relationship and therefore the less 
redundancy. 

The redundancy reflected here could be both synchronic 
and sequential. In the case of synchronic redundancy, the 
visual and audio cues presented simultaneously would have 
the same referent; the cue in one channel would be a symbol 
for the cue in the other. in the case of sequential re- 
dundancy, one element would predetermine subsequent content 
in the other . 



Analysis 



The degree of relationship for each hypothesis was 
determined by computing Pearson product moment correlation 
coefficients. The predictability of the recall of both 
opposing elements was determined by multiple correlation 
techniques using the three redundancy measures. 
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V. FINDINGS 



The Recall Patterns 



The object of the factor analysis of recall responses 
was to find a single factor which represented the general 
over-all response pattern of subjects. The principal axis 
analysis indicates a substantial first factor which 
accounts for 39 percent of the variance (Eigenvalue, 27.14; 

69 variables) . None of the other factors accounts for more 
than eight percent . 



Most of the subjects had a high loading on the prin- 
cipal factor. An arbitrary point of .60 was established 
All those subjects (N=41) with a factor loading above .60 
were grouped together for separate analysis. (See Appendix 
J.) The recall responses were computed for this qroup as 
well as for all 69 subjects. 

The computation of the various recall patterns for 
each audio-visual unit was simply a matter of totaling the 
number of subjects responding in the particular pattern. 
These are presented in Table 1 for all 69 subjects and in 
Table 2 for the factored group. 

As would be expected, there is less deviation from 
the modal response pattern for each of the audio-visual 
units in the second table. The deviants have been screened 
out by the factor analysis . 

If we look across each of the response categories, we 
can see that there is substantial variation among the audio- 
visual units within each category. In Table 2, with a 
maximum score of 41, the standard deviation of RECALL BOTH 
is 13.00; VISUAL RECALL, 17.42; And AUDIO RECALL, 12.27. 

There is some danger that the recall patterns might 
be determined by a serial position, primacy-recency, effect. 
(8) The learning of a list of nonsense syllables, for in- 
stance, is usually most rapid for the beginning and the end 
of the list. A similar effect has also been observed with 
other materials, e.g., greater recall of the first and last 
stories of a radio newscast. (31) Curves for RECALL BOTH, 
VISUAL RECALL and AUDIO RECALL over all elements for both 
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Table 1. Recall patterns by audio-visual unit for all 69 subjects 



. o 

me 







the 69 subject and 41 subject groups, are presented in 
Appendix K. The familiar U-shaped curve indicating the 
serial position effect does not appear, but there is some 
evidence of a primacy-recency effect. It can be seen in 
Table 1 that the recall of both elements of the first, 
ninth and last audio-visual units is higher than the mean 
for all units. The recall of the fifth and seventh units 
is also higher than the mean, however. Also in Table 1, 
on visual recall, the first, fifth, seventh, eighth and 
ninth elements are higher than the mean recall. Table 2 
discloses an identical result. There seems to be some 
possibility of the serial position effect, but it cannot 
account entirely for recall patterns as irregular as those 
reported. 

Coincidentally, however, all three redundancy indices 
are high at the end points. (See Appendix L.) It would 
be expected that the redundancy measures for each element 
or unit would be distributed randomly, except perhaps for 
the last few elements which might become redundant because 
of information gained from prior material. If there is a 
primacy-recency effect, given the coincidence of high 
redundancy in the first and last elements, it would work 
in favor of the hypotheses. This possibility requires a 
more circumspect interpretation of the results. 



Audio Redundancy 



The results of the measure of redundancy in the audio 
elements is reported in Table 3. The index for each word 
is given immediately above the word. It is a ratio of the 
number of subjects with the correct response to the total 
number of subjects responding to the word. 

The audio redundancy index was a summation of the 
word indices divided by the number of words deleted from 
each sentence. These appear in Table 3. It should be 
noted again that subjects were given only the words preced- 
ing the deleted word, not the entire sentence. It is likely 
that this cut the probability of correct response by about 
half. Given this limitation, the variation among the 10 
sentences is quite good, ranging from .056 for element 
seven to as high as .419 for element 10. 



Visual Redundancy 



The results of the visual redundancy measure are in 
Table 4. Subjects apparently were able to discriminate 
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Table 4. Visual redundancy index for each of the 10 
visual elements 



Visual element 


Visual redundancy index 


1 


133 


2 


113 


3 


103 


4 


83 


5 


100 


6 


90 


7 


111 


8 


33 


9 


121 


10 


134 



between the scenes on the five-point "certain-uncertain" 
scale. The lowest scoring element was the eighth with 38, 
an average of one point from the most extreme response at 
the "uncertain" pole. The highest scene totaled 134 which 
is just short of the maximum certainty score it could ha /e 
attained (140) . The majority of subjects used the entire 
scale; only 15 failed to do so. 



Audio-Visual Redundancy 



Redundancy in the audio-visual relationship is re- 
ported in Table 5, on the following page. The results 
from both versions are reported in the same table. Sub- 
jects, of course, were better able to make correct matches, 
given the opportunity to indicate more than one possibility. 
Again, there is substantial variation among the units — from 
no correct responses to unit three to 100 percent on the 
tenth unit. 
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Table 5. Redundancy in the audio-visual relationship for 
the 10 audio-visual units 



Audio-visual 
unit 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 



A-V redundancy, 
single match, N=50 

28 

2 

1 

4 

25 

3 
30 

4 

5 
30 



A-V redundancy, 
unlimited match, N=19 

18 

6 

0 

1 

18 

5 

18 

7 

8 
19 



There is a high correlation (r— .95) between the two 
versions of the measure. Since the second version, un- 
limited matching procedure, is more appropriate for our 
purposes, as indicated in Section IV, it will serve as the 
index of audio-visual redundancy in subsequent tables. 



The Relationship Between Redundancy 
and Recall 



The product moment correlations between the various 
recall patterns and the measures of redundancy are re- 
ported in Tables 6-13. Tables 6 and 7 report the recall 
correlations with audio redundancy for the entire group in 
the recall measure (n= 69) and for the subjects remaining 
after the factor analysis (N=41) . Tables 8 and 9 report 
the same information for visual redundancy. Tables 10 and 
11 report the correlation of the recall measures for both 
groups of subjects with the audio-visual redundancy measure. 



Certain correlations, other than those bearing direct- 
ly on the hypotheses, are also reported for purposes of 
later discussion. Significance levels, and the t values 
(two-tailed), are reported for all coefficients. Although 
several hypotheses were deduced from the theoretical 
rationale, the study was considered to be exploratory and 
no significance levels weire established at the outset. 

It was decided to report all findings. Furthermore, it 
was recognized that the N of only 10 worked against the 
probability of significance at conventional levels. 



Table 6. Product moment correlations of audio redundancy 
with recall patterns for 69 subjects 







r_ 


d.f . 


t R 


Recall 


both 


.35 


8 


1.07 <.40 


Audio 


recall 


.53 


9 


1.76 <.20 


Table 7. 


Product moment correlations of audio redundancy 
with recall patterns for 41 subjects 






r_ 


d.f. 


t R 


Recall 


both 


.39 


8 


1.19 <.30 


Audio 


recall 


.59 


8 


2.08 <.10 


Table 8. 


Product moment correlations of visual redundancy 
with recall patterns for 69 subjects 






r_ 


d.f. 


t R 


Recall 


both 


.63 


3 


2.28 <.10 


Visual 


recall 


.70 


8 


2.75 <.05 
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Table 9. Product moment correlations of visu al redundancy 
with recall patterns for 41 subjects 





r_ 


d.f . 


t 


R 


Recall both 


.58 


8 


2.00 


<.10 


Visual recall 


CD 

CD 

• 


8 


2.47 


<.05 



Table 10. 


Product moment 
redundancy with 


correlations of audio-visual 
recall patterns for 69 subjects 






r_ 


d.f. 


t 


R 


Recall 


both 


.76 


8 


3.34 


A 

• 

O 

ro 


Visual 


recall 


.85 


8 


4.54 


O 

. 

V 


Audio : 


recall 


.55 


8 


1.87 


<.10 



Table 11. 


Product moment 
redundancy with 


correlations of audio-visual 
recall patterns for 41 subjects 






r_ 


d.f 


. t 


R 


Recall 


both 


.70 


8 


2.76 


ID 

O 

• 

V 


Visual 


recall 


.83 


8 


4.27 


tH 

o 

. 

V 


Audio 


recall 


vH 

LO 

• 


8 


1.68 


<.20 



The Prediction of Maximum Recall 



If audio redundancy, visual redundancy and audio- 
vis ..'.1 redundancy are all used together for the prediction 
of .e recall of both elements in the audio-visual unit, 
we have the result reported in Table 12. In this table, 
the multiple correlation coefficient is reported with its 
F value and probability. 

In Table 13, the Beta weights, their F values and 
probabilities, and the partial correlation coefficients 
are reported for the three variables. 



Table 12. Multiple correlation coefficient for audio re- 
dundancy, visual redundancy and audio-visual 
redundancy with recall of both elements (high 
first factor subjects) 



Multiple r Source 


s . s . 


d.f . 


m . s . F p 


.81 Regression 


1009.49 


3 


336.50 3.94 .07 


Error 


512.51 


6 


85.42 


Total 


1522.00 


9 





Table 13. Beta weights and partial correlation coeffi- 
cients for audio redundancy, visual redundancy 
and audio-visual redundancy with recall of both 
elements (high first factor subjects) 



Beta Standard 



Weiaht 




Error 


F 


P 


Partial r 


Audio redundancy .3054 


.2708 


1.27 


.30 


• 

ro 


Visual redundancy .31 ?Z 


.2381 


1.78 


.23 


• 

4^ 

00 


Audio-visual 

redundancy ,5200 


.2716 


3 . 65 


.11 


.62 
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VI. CONCLUSION 



pn . r-P^J hiS S° in ^ U may be said that there is an aopar- 
ent relationship between redundancy as we have defined it 

n , S f|]T’ u tane °usly presented audio-visual message units 

and the recall of the elements in the message. There is 

some support for all of the hypotheses advanced in Section 

" • 11 correlations are in the hypothesized direction, 

and almost all show a high degree of relationship. As ex- 

H™a?'i I ’ 0t i al | correlations were significant at conven- 
tional levels (under the conditions of this study, a cor- 

level^° n ° f 632 iS necessar y for significance at the .05 



Hypothesis 1; Audio Recall and Audio Redundancy 



reca ll of the audio element in 

audio and visual elements are presented 
positively related to audio redundancy . 



a message where 
simultaneously is 



np*- Th f P5 edacted relationship here is strong, although 
not quite high enough for great confidence in the result. 
There is some likelihood that a correlation as hiah as <-t>e 
one oDtainea could occur by chance. For all subjects in” 
LI measurement, r=.53, p<.20; for the subjects re- 
maining after the factor analysis, r=.59, p <.10. 

The measure may have been improved by a revision of 
the scoring method for audio redundancy. It might have 
been more consistent with the rationale for the measuring 
° ^i^bt the correct response by the number of 
syllables in the response. The present system would give 

Po^ CO i reCt res P° nse " the " exactly the same weight as the 
correct response "ventilated." 



H ypothesis 2: Visual Recall and Visual Redundancy 



The rec all 

audio and 



the visual 
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More confidence may be placed in this result. For 
all subjects in the recall measure, r=.70, p<.05; for the 
subjects remaining after factor analysis, r=.66, p<.05. 

It is interesting to note the effect of the factor 
analysis on the results. The correlation of audio redun- 
dancy with audio recall is improved by the increased homo- 
geneity in the over-all pattern of recall responses, but 
n °t the correlation of visual redundancy with visual recall. 
From observation of the raw data, not included here, it 
might be concluded that audio recall was much more hetero- 
geneous than visual recall. To force out a general pattern 
over both audio and visual recall, improved the homogeneity 
of the audio pattern at the expense of the visual pattern. 
Separate factor analyses of audio and visual recall bear 
this out. The first visual recall factor accounts for 53 
percent of the variance while the first audio recall factor 
accounts for only 30 percent. The ever -all first factor, 
as reported in Section IV, is in between at 39 percent. 



Hypothesis 3. Maximum Recall and Audio Redundancy 



The recall of both audio and visual elements which 
are presented simultaneously in a message is positively 
related to audio redundancy . 

The results here are not conclusive. For the entire 
group on the recall measure, r=.35, p<.40; for the group 
after factor analysis, r=.39, p<.30. Although a fairly 
high degree of relationship obtains, it could be a chance 
occurrence. The previously mentioned heterogeneity of 
the audio recall responses undoubtedly worked against 
higher correlation. 



Hypothesis 4: Maximum Recall and Visual Redundancy 



The recall of both audio and visual elements which 
are presented simultaneously in a message is positively 
related to visual redundancy . 

The relationship of the recall of both elements in 
the audio-visual unit to visual redundancy is quite high 
and not very likely to have occurred by chance. For the 
full group of subjects on the recall measure, r=.63, 
p<.10; for the smaller group, r=.58, p<.10. 
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H ypothesis 5: — Maximum Recall and Audio-visual Redundancy 



The recall of both audio and visual elements which 
a ye p resente d simulta n eously in a messag e is positively 
relateo to redundancy in the audio-visual relationship . 



The correlation is very high, 
the recall measure, r=.76, p<.02; 
factor analysis, r=.70, p<.05. 



For the full group on 
for the group after 



3n ^ /light be suggested that the strong relationship is 
an artifact °f the procedure. if audio and visual elements 
' n fact ' redundant, the recall of one element would 
the same as recalling the other. The responses to one 

?£o the -i°^ her ^ ould be indistinguishable. Redundancy in 
the relationship, however, as measured by the matching 
procedure, assumes only an identifiable relationship be- 
tween the two channels. ^ 



In the recall scoring, responses were attributed to 

^ J; r °ii er u channel . without difficulty. For instance, one 
° he b ^ hes t audio-visuaj units in redundancy was number 
ive. The audio element wasr "The cheese maker adds a 
curdling agent to the milk." The visual element was a 
c ose-up of a man's left hand pouring a clear liquid into 
a large vat while the other hand stirred with a paddle. 

A common recall response would be to mention something 
® a curdling agent is added" in the narration coiu”" 
using the same words as the actual narration. The response 
in the visual column would be mention of a liquid beinq 
poured into the vat and being stirred. While the unit 
measures high in redundancy, the responses to the two ele- 
ments are readily distinguishable. in fact, it can be 
argued that while the degree of redundancy between audio 
and visual elements may vary, there can never be "perfect 
redundancy" as some writers suggest. (28) The visual and 
audio elements, by their nature, must elicit different 
associative responses. 



Hypothesis 6; The Prediction of Maximum Recall 



r he rec all of both audio and visual elements whi ch 
ye presented simu l taneously in a message is best pre- 
d i _ cted by a multip le regression equation based on au dio 
r e undancy, vi sual redundancy and redundancy in th e audio - 
visual relationship^ ' 1 



This is indeed true in the present case; r=.81, p=.07 
(using the recall measure based on the subjects remaining 
after factor analysis and the audio-visual redundancy 
measure based on the unlimited matching procedure) . The 
multiple correlation is higher than any of the correla- 
tions of the individual redundancy measures with the 
criterion, recall of both elements. The partial correla- 
tions indicate that each redundancy measure contributes: 
redundancy , .42; visual redundancy, .48; and audio- 

visual redundancy, .62. However, there is some question 
about the significance of the partial correlations and the 
weights of the contributing variables in the multiple re- 
gression equation. None is significant at conventional 
levels . 



The Findings and the Broadbent Construct 



These findings tentatively testify to the value of 
the Broadbent construct of the single channel utilization 
system with its short term storage capacity. 

As much as possible, the opportunity for alternation 
of attention was limited. Certain of the elements in our 
film were not registered by immediate recall. These ele- 
ments were most likely to occur at points where the mes- 
sage was most heavily loaded with information; where the 
message was least redundant . When the message becomes 
mosi. redundant, we are likely to have maximum recall — 
recall of both elements. This conforms to what would be 
expected from a limited capacity utilization system and 
the operation of a storage system, where one element may 
be stored while the other is in process through the uti- 
lization channel ar discussed in Sections II and III. 



Implications 



The findings would probably not surprise the sophis- 
ticated producer of informational audio-visual materials . 
Redundancy in the audio and visual channels is undoubtedly 
related to what might be intuitively judged as simplicity. 
Many producers of informational audio-visual materials 
would agree that content and structure ought to be simple 
and direct. Most producers of such material would also 
agree that there ought to be an interacting relationship 
between the audio and visual materials presented simul- 
taneously. 
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Nonetheless, in practice, the principles are not al- 
ways followed. The materials used in this study are cases 
in point. The newsfilm used in the pilot studies contain- 
ed a substantial proportion of only indirectly related 
audio and visual elements. The Switzerland film clip, 
while changed somewhat, was quite close to the original 
an , of course, it represented a rather complete range of 
variation on all three redundancy measures. 



The substantial number of audio and visual elements 
which never reach the utilization channel of the receiver 
would no doubt disturb the communicator, if his intention 
is to maximize information received. For all of the 
materials used in this study, there was a low ratio of 
elements received in cognition to elements transmitted. 

e ' ^he c o st s of preparing the transmitting audio-visual 
materials are high in terms of human resources. 

The present study provides a few clues to the pro- 
ducer of audio-visual materials. It suggests that redun- 
dancy is desirable, or necessary, for multiple channel 
communication where audio and visual elements are present- 
ed simultaneously, with practice, one might learn to 
intuitively build redundancy into audio and visual elements 
and into the relationship between the two elements. 

There are some obvious needs for further research. 

The necessary limitation on the number of elements which 
can be measured by immediate recall forces extremely hiqh 
correlations before we can be relatively confident that 
they are not chance occurrences. We might be willing~to 
accept lower correlations, however, if a number of repli- 
cations were performed. 

If these replications were over a variety of materials 
another purpose would be served. We would then have some 
confidence that the findings were not specific to the 
particular content of the material in this study. 

Mode of presentation should also vary. it would seem 
that the results would generalize from classroom film pre- 
sentation, to other means of audio-visual communication 
such as television and to other settings, but this assump- 
tion was not tested. * 

We might also experiment with types of response 
Recognition is probably also meaningful in this context. 
Subjects might be exposed to a long run of synchronous 
audio and visual elements and asked to note those to which 
they had been previously exposed. These responses would 




not be difficult to quantify. It might also be possible 
to devise both verbal and visual tests to indicate learn- 
ing from the various elements. The main obstacle would be 
to arrive at items of equal difficulty for all elements. 

It would be nearly impossible, to determine whether the 
difficulty of an item was inherent in the item or whether 
it reflected inadequate reception of the element . 

If the findings could stand the test of replication 
with different content, in different audio-visual media 
and under different reception conditions, and for different 
response criteria, the instruments for measuring audio 
redundancy and visual redundancy could be very useful in a 
number of situations where the evaluation of informational 
material is important. They might be used to check edu- 
cational films or short, "single concept" films much as 
programed instructional materials are validated. The 
intention would be to work toward a maximization of redun- 
dancy which, in turn, would contribute to the receipt of 
all message elements at the cognitive level. Through the 
trial and error procedure of pretesting the material, one 
would he attempting to eliminate any points where the 
capacity of the single channel utilization system might 
be exceeded. 

Television commercials might undergo the same treat- 
ment. Television news also; in a manner similar to the 
old readability formulas that used to be periodically im- 
posed on newspaper and magazine content. The* instrument 
for measuring redundancy in the audio-visual relationship 
would need to be adapted to conventional audio-visual 
presentation where elements do not match as perfectly as 
they were forced to in the experimental film. Audio and 
visual elements frequently overlap. Another problem is 
to account for pauses in the audio channel which would 
serve the same function as redundancy. 

To simplify the procedure, it might be possible to 
use only one measurement — that of the redundancy in the 
relationship between audio and visual elements. This 
measure was the best predictor of recall and the use of 
iihe audio visual redundancy measures did not contribute 

much more to the predictability. 

It now seems quite possible that audio-visual redun- 
dancy reflects redundancy in each of the two elements. 

If there is a low certainty or comprehension of the audio 
or visual elements independently, it would be unlikely 
that there could be any perceived relationship or redun- 
dancy between the two; that is, unless one contributes to 
certainty about the other, and this, of course, would be 
detected in the audio-visual redundancy measure. 
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Future research might well concentrate on this single 
measure of redundancy. Fortunately, what is called redun- 
dancy in the audio-visual relationship, is most easily 
described to the producer of audio-visual materials. It 
is the "match" of the two elements. 

In pilot studies more recent than the present work, 
subjects were asked to scale one degree of relationship 
between several pairs of audio and visual elements which 
were presented simultaneously. They were able to do so 
with a high degree of intersubject reliability. 



Summary 



We have assumed the Broadbent model of a single chan- 
nel utilization system which has a momentary storage capa- 
city. From this model, it was hypothesized that redundancy 
in and between the audio and visual channels of an audio- 
visual message would enhance the possibility of storage 
where alternation of attention is not possible. Redundancy 
was measured in the audio portion and in the visual portion 
of a film message. An index of redundancy was also taken 
for the relationship between the simultaneously matched 
audio and visual elements. Correlations of immediate re- 
call measures with the indices of redundancy supported the 
hypotheses; some, however, at a level of confidence which 
would not be tenable without replication. 



45 




REFERENCES 



REFERENCES 



1. Ball, John and Byrnes, Francis C. (Eds.)* Research, 

Principles, and Practices in Visual Communica- 
tion . Washington, D.C.: Department of Audio- 

visual Instruction of the National Education 
Association, 1960. 

2. Berlyne, D. E. Conflict, Arousal and Curiosity . New 

York: McGraw-Hill, 1960. 

3J Birdwhistell, Ray L. "Background to Kinesics," Etc. , 
1955, Vol . 13, 10-18. 

4. Broadbent, D. E. "Immediate Memory and Simultaneous 

Stimuli," Quarterly Journal of Psychology , Vol. 

9 (1957), 1-11. 

5. Broadbent, D. E. Perception and Communication . New 

York: Pergamon, 1958. 

6. Broadbent, D. E. "Successive Responses to Simultaneous 

Stimuli," Quarterly Journal of Experimental 
Psychology , Vol. 8 (1956), 145-152. 

7. Broadbent, D. E. "The Role of Auditory Localization 

in Attention and Memory Span," Journal of Ex- 
perimental Psychology , Vol. 47 (1954), 191-196. 

8. Deese, James. The Psychology of Learning . New York: 

McGraw-Hill, 1958. 

9. Deutschmann, Paul J., Barrow, Lionel C. , Jr. and 

McMillan, Anita. "The Efficiency of Different 
Modes of Communication," AV Communication Review , 
Vol. 9 (1961), 263-270. 

10. Garner, Wendell R. Uncertainty and Structure as Psy- 

chological Concepts . New York: Wiley and Sons, 

1962. 

11. Gropper, George L. "Why _Is a Picture Worth a Thousand 
Words," AV Communication Review , Vol. 11 (1963), 
75-95. 

Harrison, R. and MacLean, M. S. Facets of Facial 

Communication . Michigan State University (mimeo) , 
1965. 



12 . 



Hartman, Frank R. "Investigations of Recognition 
Learning Under Multiple Channel Presentation 
and Retention," AV Communication Review. Vol 9 
(1961), 24-43.. 

Hazard, William R. "On the Impact of Television 1 s 

Pictured News," Journal of Broadcasting . Vol 7 
(1962-63), 43-517 

Hilgard, Ernest R. Theories of Learning . 2nd edition. 
New York: Appleton-Century-Crofts, 1956. 

Holland, J. G. , and Skinner, B. F. The Analysis of 
Behavior: A Program for Self Instructio n . 

New York: McGraw-Hill, 1961 . 

Lippmann, Walter. Public Opinion . New York: Mac- 

Millan, 1922. 

May, M. A. and Lumsdaine, A. A. Learning from Films. 
New Haven: Yale University Press, 1958. 

Miller, George A., The Magical Number Seven, Plus 

or Minus Two: Some Limits on our Capacity for 

Processing Information. in David C. Beardslee 
and Michael Wertheimer (editors). Readings in 
perception. Princeton, N.J.: Van Norstrand, 



Miller, Neal E. et aL, "Graphic Communication and 
the Crisis in Education," AV Communication 
Review , Vol. 5 (1957). ' — 

Mowbray, G. H. "Simultaneous Vision and Audition: 

The Comprehension of Prose Passages with Vary- 
ing Levels of Difficulty," Journal of Experimen- 
ts Psychology. Vol. 46 (1951), 365-372. 

Mowbray, G. H. "The Perception of Short Phrases 

^resented Simultaneously for Visual and Auditory 
Reception," Quarterly Journal of Exper imental 
Psychology , Vol. 6 (1954), 86-92. 

Munsterberg, Hugo. "Studies from the Harvard Psy- 
chological Laboratory: A. Memory,- " Psychologi- 
es Review , Vol. 1 (1894), 34-38/ — 

Nelson, Stanley (Chairman of the Nomenclature Committee 
University Film Producers Association), Journal 
University Film Producers Association , 



25 



26 



27 



28 



29 



30 



31 



32 



33 



34. 



35, 



Neu, D Morgan, "The Effect of Attention Gaining 

Devices on Film-Mediated Learning," Technical 
Report SDC 269-7-9, Instructional Film Research 
R eports , Port Washington: U. S. Navy Special — 
Devices Center, 1950. 

Osgood, Charles E. "Dimensionality of Semantic Space 
for Communication Via Facial Expressions," 

J ° Urnal ° f P s ychology , Vol . 7 

Osgood, C. E., and Sebeok, T. A. "Psycholinguistics: 
a Survey of Theory and Research Problems," 
Journal of Abnormal and Social Psvchmnn v 

Supplement^*" 2 ' U954) ' M ° rt ° n PrinCe Memorial 

Sevenn, Werner J. Cue Summa tion in Multiple chann el 
^9fflnunr cat ion . (Paper prepared for the Associa- 
tion for Education in Journalism National Con- 
vention, Iowa City, Iowa, August, 1966.) 

Sumstine, David R. »a Comparative Study of Visual 

Instruction m High School," School and Soci^y. 
Vol. 7 (February 23, 1918), 2 35-2 38. L 

S wi tze rland, Educational Film Division, Walt Disney 
Productions, Burbank, California. Y 

Tannenbaum, Percy H. "Effect of Serial Position 

on Recall of Radio News Stories," Journalism 
Quarterly , Vol. 31 (1954), 319-23. 

Taylor Wilson L. "Cloze Procedure: A New Tool 

Travers, Robert M. W. Resear ch and Theory Related 

to Audiovisual In formation Transmis sion. u^j. 

1964 lty °~ Utah# Bureau of Educational Research, 

Wittich, Walter A. and Schuller, Charles F. Audio- 
visual Materials: Their Nature and Usc~ 

York: Harper & Bros., 1962 . 



Woodworth, R. s. Experimental Psycho logy 
Henry Holt, 1938, 712 . 



New York 




49 



Zuckerman, John V. "Commentary Variations: Level 

of Verbalization, Personal Reference, and Phase 
Relations in Instructional Films on Perceptual- 
Motor Tasks," Instructional Film Research Re- 
£orts. Port Washington: U. S. Navy Special 

Devices Center Technical Report SDC 269-7-4, 
1949. 



APPENDICES 



APPENDIX A 






RECALL OF AUDIO AND VISUAL ELEMENTS IN NEWSFILM 



o 



52 



OF SULeCT^CTS 
K^CA(-L/^6- 



S£C.Dl>t>S 




BHb\0 ULfO ITS 



/ i i - 1 



i| 3 | v |s|<T 



JXUlidl 

*J | 8 | 9 |/<>l a 



g 



,'Z. 



9 1 /0 1/1 1 /£• 
/3| /v| /r| 



pj_gure A1 . — Rsc 3 .ll of opposing 3udio 3nd visual elements 

in a newsfilm story about business and 
industry. From film and script used in 
the WJIM-TV /ear-end report covering 1964. 
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Figure A2 . — Recall of opposing audio and 

visual elements in a newsfilm 
story about state apportion- 
ment. From film and script 
used in the WJIM-TV year-end 
report covering 1964. 
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Figure A4. — Recall of opposing audio and visual 

elements in a newsfilm story about 
picketing at the state capitol. 

From film and script used in the 
WJIM-TV year-end report covering 
1964. 
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APPENDIX B 



RECALL OF AUDIO AND VISUAL ELEMENTS IN 
AN EDUCATIONAL FILM 
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Figure Bl. Recall of audio and visual 

elements in a film clip 
from a Walt Disney produc- 
tion entitled Switzerland. 
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APPENDIX C 

CORRELATIONS OF CERTAINTY WITH RECALL MEASURES 
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TABLE Cl 



CORRELATION OF VISUAL CERTAINTY WITH RECALL 




TABLE C2 

CORRELATION OF AUDIO CERTAINTY WITH RECALL 



Recall Patterns 


r 


Recall, both opposing audio and visual elements 


-.22 


No recall, either opposing element 


.29 


Recall, audio element only 


00 

n-i 

• 


Recall, visual element only 


-.32 


Total recall, visual (recall both plus visual 




only) 


o 

i-i 

• 

i 


Total recall, audio (recall both plus audio 




only) 


-.37 
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TABLE C3 



CORRELATION OF CROSS-CHANNEL RELATIONSHIP WITH RECALL 



Recall Patterns 

Recall, both opposing audio and visual elements 
No recall, either opposing element 
Recall, audio element only 
Kscsii, visual element; only 



r 



.38 



-.20 



-.OS 

« 



recall, yisual (recall both plus visual 
only) 

Total recall, audio (recall both plus audio 
only) 
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APPENDIX E 



RECALL INSTRUMENT 



COLLEGE OF COMMUNICATION ARTS 



Michigan State University 



INSTRUCTIONS 



You have just seen a sound film about cheese-making 
in Switzerland. On the following pages you will find a 
form for writing as much as you can remember about the 
narration and the visual. Write all the details you can 
recall from the narration in the right hand column. Write 
all the details you can recall from the visual in the left 
hand column. 

Do not be concerned about the order of the items in 
either column; put them down as they come to you. It is 
not necessary to match the narration with the visual. 

The two columns, as you fill them in, do not have to have 
any relationship to each other. 

Please try your best to write down all of the elements 
that you can recall from both visual and narration. 
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APPENDIX F 



RECALL SCORE SHEET 
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APPENDIX G 

AUDIO REDUNDANCY INSTRUMENT 
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INSTRUCTIONS, AUDIO REDUNDANCY 



In a moment I will distribute a small booklet. Please 
do not open it until I tell you. PASS OUT BOOKLETS. Please 
write your name in the upper right hand corner of the first 
page. 

On each of the pages inside this booklet is a part of 
a sentence. You are to read the word or words quickly and 
try to guess what the next word would be. Guess quickly 
and write the word on the blank provided. The word you 
supply is not necessarily the last word of a sentence, in 
fact in most cases it will not be. The length of the blank 
has nothing to do with the length of the word that is miss- 
ing. Each blank is the same length no matter how long the 
next word is. Stop writing when I say "stop." I will then 
read the complete sentence so you will know what the miss- 
ing word was--you are not to write it down at this point, 
however. Once I have finished reading, go immediately to 
the next page and try to guess the word in that sequence. 
Make sure you turn only one page at a time. 

We will go through ten sentences in this manner. I 
will repeat the instructions. REREAD ALL AFTER "PASS OUT 
BOOKLETS." Are there any questions? Please turn over the 
first page and start. READ THE SENTENCE SILENTLY, ALLOW 
TIME TO FILL IN A WORD, THEN SAY . . . Stop! 

READ THE SENTENCE ALOUD. Now turn over to the next 
page and try to guess the word. 
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Close by, the official cheese maker 
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The amount of 




Its owner will receive cheese 
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The cheese maker adds a curdling agent to the 
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The mixture is 




Now after a proper interval. 
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Then the 
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By the way. 



it's not the ventilated 
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Dated and labeled, the cheeses 
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APPENDIX H 



VISUAL REDUNDANCY INSTRUMENT 



COLLEGE OF COMMUNICATION ARTS 



Michigan State University 
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INSTRUCTIONS 



In a moment you will see silent scenes from a short 
f i lm ; are interested in finding out how well you under- 

stand the scenes. You will be asked to rate the scenes 
in accordance with the degree of certainty you have in 
your understanding of the scene. 

Again, you are to rate the scenes in terms of the 
degree of certainty about your comprehension of what you 



If the scene is quite clear to you— if you are certain 
that you comprehend what you saw, check the blank next to 
the word CERTAIN: 



CERTAIN l/: 



UNCERTAIN 



*5 t ^ ie scene i s fairly clear — if you are fairly cer- 
tain you comprehend what you saw, check the second bliHkT 

CERTAIN : : : : UNCERTAIN 



For " not sure ," check the middle blank; 

CERTAIN : : _U^ : : UNCERTAIN 

For fairly uncertain , check the fourth blank: 
CERTAIN : : : iX UNCERTAIN 



If you are completely uncertain about what you have 

S u Sn r~ic , scene is ver Y unclear or confusing to you, 
check the blank next to the word UNCERTAIN: 



CERTAIN : ; ; : l^UNCERTAIN 



The screen will be blank for ten seconds after each 
scene. Make your rating of the scene during this period. 



CERTAIN : : : : UNCERTAIN 

CERTAIN : : : : UNCERTAIN 

CERTAIN : 2 2 : UNCERTAIN 



CERTAIN 


: : 2 2 UNCERTAIN 


CERTAIN 


: : 2 2 UNCERTAIN 


CERTAIN 


: : : 2 UNCERTAIN 


CERTAIN 


: : : 2 UNCERTAIN 


CERTAIN 


22:2 UNCERTAIN 


CERTAIN 


2222 UNCERTAIN 


CERTAIN 


2222 UNCERTAIN 



APPENDIX I 



AUDIO-VISUAL REDUNDANCY INSTRUMENT 



[ SECOND ADMINISTRATION! 



COLLEGE OF COMMUNICATION ARTS 
Michigan State University 

INSTRUCTIONS 

This will take just a few minutes. You will be asked 
to match motion picture scenes with sentences which might 
be in a script for the film. The scenes and the sentences 
have been mixed up. So that you are familiar with the 
sentences, we will read them over twice. Please turn the 
page and read along silently as they are read aloud. 

In a moment we will look at the scenes you are to 
nistch with the sentences. You will be shown a scene only 
once. The screen will go blank after each scene. As soon 
as it does, you will be reminded of what number it is. 

You are to write the number opposite the sentence that 
most closel y corresponds with that scene and any other 
sente nce that might also fit the scene . Again, write the 
number opposite the sentence that seems most closely related 
to the s cene and any other sentence that might possibly 
match the scene . You therefore, might have a particular 
scene number written in front of more than one sentence. 
Guess, if you see no obvious choice. You may also use 
a sentence more than once. in other words, some sentences 
may have more than one scene number on the blank next to 
it. Other sentences may not be used at all. You will have 
about 40 seconds to make your choices. At the end of this 
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time, you will be warned so that you can finish and look 
up at the screen for the next scene. Do not change your 
final choices for a scene after the next one starts. 
Since the scenes last only about four seconds, you will 
have to pay close attention. 



Its owner will receive cheese and butter 
in proportion. 



Now after a proper interval, the curds are 
removed. 



The amount of milk given by each cow is 
measured. 



Close by, the official cheese maker sets 
up shop. 



Then the cheese is pressed into round 
wooden molds. 



The mixture is heated in large open vats. 



Dated and labeled, the cheeses are stored 
on shelves. 



Through the summer months, cows graze in 
upland pastures. 



By the way, it's not the ventilated type 
commonly known as Swiss cheese. 



The cheese maker adds a curdling agent 
to the milk. 
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[ FIRST ADMINISTRATION! 



COLLEGE OF COMMUNICATION ARTS 
Michigan State University 

INSTRUCTIONS 

This w.i 11 take just a few minutes. You will be asked 
to match motion picture scenes with sentences which might 
be in a script for the film. The scenes and the sentences 
have been mixed up. So that you are familiar with the 
sentences, we will read them over twice. Please turn the 
page and read along silently as they are read aloud. 

In a moment we will look at the scenes you are to 
match with the sentences. You will be shown a scene only 
once. The screen will go blank after each scene. As soon 
as it does, you will be reminded of what number it is. 

You are to write the number opposite the sentence that 
most closely corresponds with that scene . Again, write 
the number opposite the sentence that seems most closely^ 
related to the scene . Guess, if you see no obvious choice. 
You may use a sentence more than once. In other words, 
some sentences may have more than one scene number on the 
blank next to it. Other sentences may not be used at all. 
You will have about 60 seconds to make your choice. At 
the end of this time, you will be warned so that you can 
finish and look up at the screen for the next scene. Do 
not change your final choice for a scene after the next 
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one comes up. 
seconds, you 



Since the scenes only last about four 
ill have to pay close attention. 
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Its owner will receive cheese and butter 
in proportion. 



Now after a proper interval, the curds are 
removed. 



The amount of milk given by each cow is 
measured. 



Close by, the official cheese maker sets 
up shop. 



Then the cheese is pressed into round 
wooden molds. 



The mixture is heated in large open vats. 



Dated and labeled, the cheeses are stored 
on shelves. 



Through the summer months, cows graze in 
upland pastures. 



By the way, it's not the ventilated type 
commonly known as Swiss cheese. 



The cheese maker adds a curdling agent to 
the milk. 
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APPENDIX J 



PRINCIPAL FACTOR LOADINGS 



Table Jl. 



Factor loadings on the principal factor in 
the analysis of recall responses (asterisk 
indicates subjects with loading of .60 or 
over) . 



Subject 


Factor 


Subject 


Factor 


Subject 


Factor 


number 


loadinq 


number 


loadinq 


number 


loadinq 


1 


.230 


24* 


.600 


47* 


.906 


2 


.279 


25 


.444 


48 


.561 


3* 


.642 


26 


.525 


49* 


.805 


4* 


.661 


27* 


.625 


50* 


.641 


5* 


.602 


28* 


.640 


51* 


.791 


6* 


.636 


29 


.399 


52* 


.685 


7* 


.677 


30 


.577 


53* 


.821 


8* 


.771 


31* 


.692 


54 


.544 


9 


.567 


32* 


.775 


55* 


.656 


10* 


.619 


33 


.430 


56* 


.633 


11 


.476 


34* 


.721 


57 


.585 


12* 


.630 


35 


.557 


58* 


.804 


13* 


.712 


36* 


.706 


59 


.569 


14 


.596 


37* 


.669 


60* 


.639 


15 


.597 


38 


.491 


61* 


.716 


16* 


.625 


39 


.426 


62* 


.828 


17* 


.711 


40* 


.687 


63 


.432 


18* 


.638 


41* 


.816 


64 


.523 


19 


.117 


42 


.069 


65 


.323 


20 


.362 


43 


.437 


66* 


.798 


21* 


.806 


44 


.584 


67 


.442 


22 


.444 


45* 


.659 


68* 


.815 


23* 


.717 


46* 


.644 


69* 


.896 
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RECALL CURVES 
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Figure Kl. — Recall 
curve for the subjects 
recalling both ele- 
ments of each of the 
10 audio-visual units 
(N=69) . 



Figure K2. — Recall 
curve for subjects 
recalling both audio 
and visual elements 
in each of the 10 
audio-visual units 
(N=41 ) . 
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Figure K3. — Recall 
curve for subjects 
recalling the visual 
element in each of the 
10 audio-visual units 
(N=41; . 
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Figure K4. — Recall 
curve for subjects 
recalling the audio 
element in each of the 
10 audio-visual units 
(N=41) . 
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Figure K5. — Recall curve for the subjects 

recalling the visual elements in 
each of the 10 audio-visual unit 
(N=69) . 
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Figure K6. — Recall curve for the subjects 

recalling the audio element in 
each of the 10 audio-visual 
units (N=69) . 
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REDUNDANCY CURVES 
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Figure Li. — Visual 
redundancy by audio- 
visual unit. 
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Figure L2. — Audio 
redundancy by audio- 
visual unit. 
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Figure L3. — Audio-visual redundancy by audio- 
visual unit. 
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